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ROBUST DETECTION OF A REFERENCE IMAGE DURING MAJOR PHOTOMETRIC 

TRANSFORMATIONS 

Preamble to the Specification 

Field Concerned, Problem Posed 

[0001] The present invention pertains to a method and a system for identifying 
a specific image and/or a specific audiovisual sequence within any flow of images or of 
audiovisual sequences, and in particular with the prospect of being able to identify a 
proprietary image within the flow and/or of being able to identify, preferabiy in real 
time, a plurality of proprietary audiovisual sequences within the flow. 

[0002] In the field of monitoring protected audiovisual contents, one of the 

problems concerns the identification of a proprietary video sequence during its 
broadcast. Since a video sequence is a stream of images, the solution to the problem 
implies being able to detect and identify a particular image (called a reference image) 
in real time, while being robust during a certain number of photometric 
transformations, which may affect the image during its broadcast. 

[0003] The solution to this problem: 

must be robust with small differences in features, 

must make fast calculations possible, 

must have a strong discriminating power. 

Solution 

Tlie Method According to the Present Invention 

[0004] The method comprises the step of calculating, for each image, an index 

appearing in the form of a characteristic vector, encoding the content of the image. 
The index calculation process is hereinafter called the indexing process. 

[0005] The method comprises the following steps using the indexing process: 
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the step of calculating a reference Index, using the Indexing process for 
the specific image, or 

the step of extracting reference indexes from the specific audiovisual 
sequence so as to form a reference set of reference indexes. 

[0006] Characteristic reference indexes of the specific image and/or of the 
specific audiovisual sequence are thus obtained. 

[0007] The method additionally comprises the step of calculating an Index for 
current images of the flow, using the indexing process for the current Images of the 
flow. The Index thus calculated is hereinafter called the current Index. 

[0008] The method comprises the step of comparing the reference indexes with 
the current index of the current image of the flow monitored. It is thus possible to 
detect a specific image within a flow with great precision and extremely fast, while 
being robust during major photometric alterations. 

Indexing Process 

[0009] Preferably, according to the present invention, the method Is such that, 
for calculating an index of an image, and in particular a reference index and/or a 
current index. It comprises the step of resampling the Image as an image with fixed 
dimensions in advance. The resampled Image Is hereinafter called the normalized 
image. 

[0010] If the Image is a color image comprising levels of colors, the method 
additionally comprises the step of converting the levels of colors of the Image to be 
resampled to levels of gray beforehand. 

[0011] The normalized image is represented by a matrix of pixel values after 
discrete quantization of the pixel values. 

[0012] The method additionally comprises the step of arranging the values 

according to a predetermined running order of the positions in the matrix, and in 
particular by concatenating the values of each line of the matrix in the form of a 
characteristic vector. This vector forms the index. 
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[0013] Preferably, according to the present invention, the method additionally 
comprises the step of calculating the discrete entropy of the distribution of the values 
of the reference index or the current index. The entropy is hereinafter called the 
reference marginal entropy or the current marginal entropy. 

[0014] The comparison time is optimized by proceeding In this way. 

[0015] The marginal entropy value may be added to the index. 

Calculation of an Index Comparison Distance 

[0016] Preferably, according to the present invention, the Indexes appear in the 

form of ordered and finite sets of values. These values are Identified, In the reference 
index and the current Index, by a system of coordinates. 

[0017] The method additionally comprises the step of defining, for a given 
coordinate of the system of coordinates, a pair of values: 

of which the first value is the value appearing in the reference index 
associated with the given coordinate, and 

of which the second value Is the value appearing in the current Index 
associated with the given coordinate. 

[0018] The method additionally comprises the step of calculating the 
bidlmensional histogram of the pairs of values obtained for all the coordinates of the 
system of coordinates of the reference index and of the current Index. 

[0019] The method additionally comprises the step of calculating the discrete 
entropy of the bidlmensional histogram, hereinafter called the entropy of the 
bidlmensional histogram. 

[0020] The method additionally comprises the step of calculating a comparison 

distance between a reference index and a current index, forming the ratio between the 
sum of the reference marginal entropy and of the current marginal entropy reduced by 
the entropy of the bidlmensional histogram as the numerator and the sum of the 
reference marginal entropy and of the current marginal entropy as the denominator. 



GRY-134US 



PATENT 



- 4 - 

Extraction of Reference Indexes 

[0021] Preferably, according to the present invention, tlie method is such that, 
to extract reference indexes of the specific audiovisual sequence from the specific 
audiovisual sequence, it additionally comprises the step of initializing a reference set 
containing the reference indexes of specific images. This set is initialized with the 
reference index of the first specific image of the specific audiovisual sequence. The 
reference index of the first specific image of the specific audiovisual sequence 
constitutes the first reference index of the reference set. 

[0022] The method additionally comprises: 

(a) the step of calculating, for each specific image of the specific 
audiovisual sequence, a temporary current index and of calculating a comparison 
distance between the temporary current index and the last reference index added to 
the reference set, 

(b) the step of comparing the comparison distance between the 
temporary current Index and the last reference index added to the reference set to a 
predetermined threshold SE, 

(c) the step of adding the temporary current index to the reference set if 
the comparison distance exceeds the predetermined threshold SE. 

[0023] The temporary current index becomes the last reference index of the 
reference set. 

[0024] The method additionally comprises the step of repeating the steps (a) 
through (c) up to the end of the specific audiovisual sequence. 

Detection 

[0025] Preferably, according to the present invention, the method is such that, 

for comparing the reference indexes with the current Index of the current image of the 
monitored flow, it additionally comprises the step of comparing the comparison 
distance to a predetermined threshold SF so that the specific image is detected within 
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any flow of images when the comparison distance between the reference index of the 
specific image and the current index is less than the predetermined threshold SF. 

[0026] According to another embodiment variant of the present invention, the 
method is more particularly designed to detect a specific audiovisual sequence within 
any flow of audiovisual sequences. Preferably, in the case of this embodiment variant, 
the method comprises: 

(a) the step of initializing a variable T at -1 and of initializing a variable D 

at 0, 

(b) the step of calculating, for each reference index of the reference set, 
the comparison distance between the reference index of the reference set and the 
current index. 

[0027] If the comparison distance thus calculated is less than a predetermined 

threshold SD, the variable D is increased by one. This condition is hereinafter called 
the condition for detecting reference indexes. 

[0028] The moment when the first reference index of the reference set of the 
specific audiovisual sequence meets the detection condition is hereinafter called the 
moment of the first detection. 

[0029] The method additionally comprises the following steps: 

(c) the step of assigning to the variable T the time elapsed since the 
moment of the first detection if the variable D is different from zero, 

(d) the step of repeating step (b) until the variable D reaches the 
predetermined threshold SD, or of repeating step (a) if the variable T exceeds the 
predetermined threshold ST, 

(e) the step of detecting the specific audiovisual sequence if the variable 
D reaches the predetermined threshold SD. 
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The System According to the Present Invention 

[0030] The system comprises: 

first calculation means for calculating a reference index for the specific 
image, using an indexing process, or 

first computer analysis means for extracting reference indexes from the 
specific audiovisual sequence, so as to form a reference set of reference indexes. 

[0031] The reference index appears in the form of an ordered and finite set of 
values, and in particular in the form of a characteristic vector, encoding the content of 
the specific image. The combination of the technical features results in that a reference 
index characteristic of the specific Image and/or of the specific audiovisual sequence is 
thus obtained. The system comprises: 

reception means for receiving the flow of images or audiovisual 
sequences comprising at least one specific image and/or at least one specific 
audiovisual sequence, 

computer processing means for digitizing the flow of images or 
audiovisual sequences. 

[0032] The system additionally comprises second calculation means for . 
calculating a current index for the current images of the flow, using the indexing 
process for the current images of the flow. The current index appears in the form of an 
ordered and finite set of values, and in particular in the form of a characteristic vector, 
encoding the content of the current image. The system additionally comprises 
comparison means for comparing the reference index of the specific image with the 
current index of the current image of the monitored flow. The combination of the 
technical features results in that the system makes it possible to detect a specific 
image within a flow with great precision and extremely fast, while being robust during 
major photometric alterations. 
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Indexing Process 

[0033] Preferably, according to the present invention, the first calculation means 
for calculating a reference index of a specific image comprise: 

sampling means for resampling the specific image as a resampled 
specific image with fixed dimensions in advance, 

means for the discrete quantization of the pixel values of the resampled 
specific image. 

[0034] After discrete quantization, the resampled specific image is represented 
by a matrix of the pixel values. 

[0035] The first means for calculating the reference index of a specific image 

additionally comprise sequencing means for arranging the pixel values according to a 
predetermined running order of the positions in the matrix, and in particular by 
concatenating the values of each line of the matrix in the form of a characteristic 
vector. The reference index is thus obtained. 

[0036] If the specific image is a color image comprising levels of colors, the 
system additionally comprises conversion means for converting the levels of colors of 
the specific image to be resampled to levels of gray beforehand. 

[0037] Preferably, according to the present invention, the first calculation means 
additionally comprise reference processing means for calculating the discrete entropy of 
the distribution of the values of the reference index. This entropy is hereinafter called 
the reference marginal entropy. 

[0038] It is thus possible to optimize the comparison time. It is possiMe to add 
this reference marginal entropy value to the reference index. 

[0039] Preferably, according to the present invention, the system is such that 

the second calculation means for calculating a current index of a current image 
comprise: 
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sampling means for resampling the current image as a current image 
with fixed dimensions in advance, 

means for the discrete quantization of the pixel values of the current 

image. 

[0040] After discrete quantization, the resampled current image is represented 
by a matrix of the pixel values. 

[0041] The second calculation means for calculating a current index of a current 
image additionally comprise sequencing means for arranging the pixel values according 
to a predetermined running order of the positions in the matrix, and in particular by 
concatenating the values of each line of the matrix in the form of a characteristic 
vector. The current index is thus obtained. 

[0042] If the current image is a color image comprising levels of colors, the 

system additionally comprises conversion means for converting the levels of colors of 
the current image to be resampled to levels of gray beforehand. 

[0043] Preferably, according to the present invention, the second calculation 

means additionally comprise current processing means for calculating the discrete 
entropy of the distribution of the values of the current index. This entropy is 
hereinafter called the current marginal entropy. 

[0044] It is thus possible to optimize the comparison time. It is possible to add 
this current marginal entropy value to the current index. 

Calculation of an Index Comparison Distance 

[0045] Preferably, according to the present invention, each reference index and 
each current index appear in the form of ordered and finite sets of values. These 
values are identified, in the reference Index and the current index, by a system of 
coordinates. The system is such that it additionally comprises third calculation means 
for defining, for a given coordinate of the system of coordinates, a pair of values, of 
which the first value is the value appearing in the reference index associated with the 
given coordinate, and of which the second value is the value appearing in the current 
index associated with the given coordinate. The third calculation means malce it 
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possible to calculate the bidimensional histogram of the pairs of values obtained for all 
the coordinates of the system of coordinates of the reference Index and of the current 
Index. 

[0046] The third calculation means also make it possible to calculate the discrete 
entropy of the bidimensional histogram, hereinafter called the entropy of the 
bidimensional histogram. 

[0047] The third calculation means also make it possible to calculate a 

comparison distance between a reference index and a current index, forming the ratio 
between the sum of the reference marginal entropy and of the current marginal 
entropy reduced by the entropy of the bidimensional histogram as the numerator and 
the sum of the reference marginal entropy and current marginal entropy as the 
denominator. 

Extraction of Reference Indexes 

[0048] Preferably, according to the present Invention, the system is such that 

for extracting from the specific audiovisual sequence, made up of specific images, the 
reference Index of the specific audiovisual sequence. It additionally comprises fourth 
calculation means. These fourth calculation means use a calculation algorithm 
comprising a step of initializing a reference set containing the reference indexes of the 
specific images. The reference set is initialized with the reference index of the first 
specific image of the specific audiovisual sequence. The reference index of the first 
specific image of the specific audiovisual sequence constitutes the first reference index 
of the reference set. The calculation algorithm additionally comprises: 

(a) the step of (i) calculating, for each specific Image of the specific 
audiovisual sequence, a temporary current Index and (ii) of calculating a comparison 
distance between the temporary current index and the last reference index added to 
the reference set, 

(b) the step of comparing the comparison distance between the 
temporary current index and the last reference index added to the reference set to a 
predetermined threshold SE, 
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(c) the step of adding the temporary current index to the reference set, if 
the comparison distance exceeds the predetermined threshold SE. 

[0049] The temporary current index becomes the last reference index of the 
reference set. The calculation algorithm additionally comprises the step of repeating 
the steps (a) through (c) up to the end of the specific audiovisual sequence. 

Detection 

[0050] Preferably, according to the present invention, the system is such that 

the third calculation means compare the comparison distance between each reference 
index and the current index of the current image of the monitored flow to a 
predetermined threshold SF in such a way that the specific image Is detected within 
any flow of Images when the comparison distance between the reference index of the 
specific image and the current index Is less than the predetermined threshold SF. 

[0051] According to another embodiment variant of the present invention, the 

system is more particularly designed for detecting a specific audiovisual sequence 
within any flow of audiovisual sequences. In this case, the system comprises 
Initialization means for loading the value -1 In a first register T and the value 0 in a 
second register D. 

[0052] In the case of this variant, the system additionally comprises fifth 
calculation means for calculating, for each reference index of the reference set, the 
comparison distance between the reference index of the reference set and the current 
index. 

[0053] If the comparison distance thus calculated Is less than a predetermined 
threshold SD, the second register D is increased by one. This condition is hereinafter 
called the condition for detecting reference indexes. 

[0054] The moment when the first reference index of the reference set of the 
specific audiovisual sequence meets the detection condition is hereinafter called the 
moment of the first detection, 

[0055] The fifth calculation means are equipped for loading in the first register T 

the time elapsed since the moment of the first detection if the value stored in the 
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second register D is different fronn zero. The fifth calculation means are equipped (i) 
for repeating the calculation of the comparison distance until the value stored in the 
second register D reaches the predetermined threshold SD, or (ii) for repeating the use 
of the initialization means if the value stored in the first register T exceeds a 
predetermined threshold ST. 

[0056] In such a way that the specific audiovisual sequence is said to be 
detected if the stored value of the second register D reaches the predetermined 
threshold SD. 

. Mathematical Precisions about tlie Nature of tlie Comparison Function Used in 
tlie Present Invention 

[0057] The method that is the subject of the present Invention makes it possible 

to detect proprietary audiovisual sequences within a video flow to be analyzed. This 
method is based on the existence of a comparison distance making it possible to 
compare any two images. 

[0058] This function has the property of returning a low value, close to 0, when 

the two images are different and returning a high value when the two images are 
superimposable, even In the presence of significant photometric transformations 
between the two images, i.e., profoundly changing the pixel values of an image. 

[0059] Section 1 defines the function used for the comparison, and section 2 
shows why the detection takes place under the given, even difficult, conditions. 

1. Mutual Information Between Two Random Variables 

[0060] Originating from the field of statistics, the concepts mentioned below are 
known and can be found In technical works presenting the foundations of the theory of 
communications, for example. In Information Theory, by Robert B. Ash, Dover 
Publications Inc. 
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1.1 Concept of Discrete Entropy 

[0061] If X is a random variable having discrete values {x_l, ... x_n> witli the 
corresponding distribution of probabilities {p_l/ ... p_n> (i.e., p(X = x_l) = p_lr ...^ 
p(X = x_n) = p_n), the discrete entropy of X is by definition: 

H(X) = -Zpx_/xlogfpx_/; 

1.2 Concept of Joint Discrete Entropy 

[0062] If X is a random variable having discrete values {x_l/ ... x_n> with the 
corresponding distribution of probabilities -Cpx_l, ... px_n>; 

If Y is a random variable having discrete values ■Cy_l/ ... y_n> with the 
. corresponding distribution of probabilities -Cpy_l/ ... py_n>; 

If the joint random variable Z = (X, Y), having by definition the discrete 
values {(x_l/ y_l), — / (x_n^ y_n)> provided with the corresponding distribution of 
probabilities <pz_ll, ... pz_nn> 

in such a way that p(X = x_l, Y = y_l) = pz_ll, p(X = x_n^ Y = 
V-n) = pz_nn), 

then the joint discrete entropy of Z = (X, Y) is by definition 

H(Z) = H(X, Y) = -Z,,pzJjx\og(pxJj) 

1.3 Concept of Conditional Discrete Entropy 

[0063] If X is a random variable having discrete values <x_l, ... x_n> with the 
corresponding distribution of probabilities {px_l, ... px_n}; 

If Y is a random variable having discrete values {y_l/ ... y_n> with the 
corresponding distribution of probabilities -Cpy_l/ ... py_n>; 

If the conditional discrete random variable W = (XjY), declaring "X 
knowing Y", having by definition discrete values -C(x_l|y_l), (x_n|y_n)} 
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provided with the corresponding distribution of probabilities -Cpw_ll, ... pw_nn>, in 
such a way that p(X = x_l|Y = y_l) = pw_ll^ p(X = x_n|Y = y_n) = 
pw_nn), then the conditional discrete entropy of W = (X| Y) is by definition 

H(Z) = H(X, Y) = 'Zi,pw_iJxlog(pwJJ) 

It is not difficult to prove that H(X,Y) = H(X|Y) + H(Y). 

1.4 Concept of Mutual Information 

[0064] The mutual Information MI(X,Y) between two random variables X and Y 
is by definition: 

MI(X,Y) = H(X) + H(Y) - H(X,Y) 

MI(X,Y) = H(X) - H(X|Y) 

MI(X,Y) = H(Y) - H(Y|X) 

1.5 Concept of Normalized Mutual Information 

[0065] The normalized mutual Information NMI(X,Y) between two random 
variables X and Y is by definition given by the following formula: 

NMI(X,Y) = MI(X,Y) / (H(X) + H(Y)) 

[0066] The concept of comparison distance entering into the definition of the 
technical features of the present invention corresponds to NMI. 

1.5.1 Case of Independent Variables 



[0067] If X and Y are independent, then by definition H(X| Y) = H(X), and thus 

NMI(X,Y) = 0. 



GRY-134US 



PATENT 



- 14 - 

1.5.2 Case of Functionally Linked Variables 

[0068] If Y = f(X), then H(Y|X) = H(ff(X)|X) = O, because the value of the 
random variable f(X) Is entirely determined by the knowledge of X. Referring to the 
definition of mutual information, the remarkable simplification is obtained: 

MI(X,Y) = H(Y) = H(f(X)) 

therefore, 

NMI(X,Y) = H(f(X)) / (H(X) + H(f(X))) 

2. Detection of Images by Normalized Mutual Information 

2.1 Images, Histograms and Random Variables 

[0069] The image detection method according to the present invention is based 

on the definitions and properties that were just explained. 

[0070] In fact: 

• The normalized histogram of the levels of gray of an image I, 
obtained by calculating the histogram of the values taken by I(x) for x having all the 
possible positions in the image, is a distribution of discrete probabilities making it 
possible, by extension, to define "the entropy of an image," see paragraph 1.1. 

• The normalized joint histogram of the levels of gray of two images 
II and 12 of the same size, obtained by calculating the bidimensional histogram of the 
values taken by (Il(x), I2(x)) for x having all the possible positions in the image II, 
Is a bidimensional distribution of probabilities making it possible, by extension, to 
define "the joint entropy between two images," "the conditional entropy between two 
images/' "the mutual information between two images," "the normalized mutual 
information between two images," respectively, see paragraphs 1.2, 1,3, 1.4 and 1.5, 
respectively. 

[0071] Thus, the comparison distance used for proceeding with detections is 

NMI(X^Y), where X and Y are two images. 
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2.2 Detection Criterion 

[0072] To detect a specific image, the comparison distance NMI(X,Y) is 
compared to a predetermined threshold, fixed in advance. If the distance between the 
current image, which is the candidate for the detection, and the reference image is less 
than this threshold, the current image is declared "recognized" or detected. 

2.3 Robustness During Photometric Transformations 

[0073] In theory, the image to be detected is the exact copy of the reference 
image. IHowever, in practice, the image to be detected generally has passed through a 
noisy transmission channel, Hertzian waves, television receiver, satellite, 
magnetoscope, etc. This noise may be expressed either as high-frequency noise in the 
image but also as a low-frequency deformation of the signal, change in contrast or in 
brightness, saturation, etc. 

[0074] Thus, it is essential that the detection technique be robust during these 
photometric changes. 

[0075] The distance NMI(X,Y) has the advantage of not directly comparing the 

pixel values of two images (a simple approach used, for example, in a correlation 
distance but unusable in practice because of its lack of robustness). On the other 
hand, the distance NMI(X,Y) has the advantage of calculating the capacity to predict 
the pixel values of X knowing those of Y, without a particular hypothesis about the 
nature of the photometric transformation linking X and Y. 

[0076] In other words, the distance NMI(X,Y) remains minimal if X and Y 
correspond with one another geometrically, even if their intensity surfaces are not 
directly superimposable. Therefore, the detection is extremely robust during 
photometric changes. 

Detaiied Description 

[0077] Other features and advantages of the present invention will become 

apparent after reading the description of embodiment variants of the present invention 
given by way of indicative and nonlimiting example and 
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Figure 1 which schematically shows any flow 3 of image[s] 6 containing a 
specific image 11, and in particular a proprietary image 4 that should be detected. 

Figure 2 which schematically shows the technical means making it possible to 
calculate a reference index 10 of a specific image 11, 

Figure 3 which schematically shows the form in which a reference index can 
appear, 

I 

Figure 4 which schematically shows the indexing process 39 making it possible 
to calculate an index 8, and in particular a current index 14 in order to compare it to a 
reference index 10, 

Figure 5 which schematically shows a flow 3 of audiovisual sequences 7 
containing a specific audiovisual sequence 2, and in particular a proprietary audiovisual 
sequence 4, 

Figure 6 which schematically shows the technical means making it possible to 
calculate a reference set 30 made up of reference indexes 10 from a specific 
audiovisual sequence 2, 

Figure 7 which schematically shows the form in which a reference index 10 of a 
reference set 30 can appear. 

Figure 8 which schematically shows the indexing process 39 making it possible 
to calculate an index 8, and in particular a current index 14 of a current image 13 of 
any audiovisual sequence 7, in order to compare it to a reference index 10, 

Figure 9 which schematically shows an embodiment of a process for indexing a 
specific image 11 in the case of the first embodiment variant as in the case of second 
embodiment variant. 

Figure 10 which schematically shows an embodiment of a process for indexing a 
current image 13 in the case of the first embodiment variant as in the case of the 
second embodiment variant. 
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Figure 11 which schematically shows an embodiment of the pairs of values 25, 
26 making up a reference Index 21a and a current index 21b, 

Figure 12 which schematically shows the technical means making it possible to 
calculate a comparison distance 29, 

Figure 13 which schematically shows an embodiment of the technical means 
making it possible to extract the reference Indexes 10 and to form a reference set 30 in 
the case of the second embodiment variant. 

Figure 14 which schematically shows an embodiment of the technical means 
making it possible to detect a specific audiovisual sequence 2 In the case of the second 
embodiment variant. 

Figure 15 which schematically shows. In the case of the second embodiment 
variant, the flow chart of the algorithm making it possible to detect a specific 
audiovisual sequence 2 using the technical means described with reference to Figure 
12. 

First Embodiment Variant 

[0078] In the case of the first embodiment variant of the present invention 
which will now be described with reference to Figures 1, 2, 3 and 4, the system is 
designed for identifying a specific image 11 within any flow 3 of images 6. The 
objective is to Identify a proprietary Image 4 in the flow 3. 

[0079] In the case of this first embodiment variant, the system comprises first 
calculation means 38 for calculating a reference index 10 for each specific image 11, 
using an Indexing process 39. Such an Indexing process 39 shall be described In detail 
below. 

[0080] The reference index 10 (Figure 3) appears in the form of an ordered and 

finite set 21a of values 20a, and in particular In the form of a characteristic vector 9a, 
encoding the content of the specific image 11. A reference index 10 characteristic of 
the specific image 11 is thus obtained. 
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[0081] The system additionally comprises reception means 41 for receiving the 
flow 3 of images 6 capable of comprising at least one specific image 11. The system 
additionally comprises computer processing means 42 for digitizing the flow 3 of 
images 6. The system additionally comprises second calculation means 43 for 
calculating a current index 14 for the current images 13 of the flow 3, These second 
calculation means 43 calculate the current index 14 using an indexing process 39 
comparable to the one used for the calculation of the reference indexes 10 of the 
specific images 11. Just as the reference index 10, the current index 14 appears in the 
form of an ordered and finite set 21b of values 20b, and in particular in the form of a 
characteristic vector 9b encoding the content of the current image 13. 

[0082] The system additionally comprises comparison means 44 for comparing 
the reference index 10 of the specific image 11 with the current index 14 of the current 
image 13 of the monitored flow 3. It is thus possible to detect a specific Image 11 
within a flow 3 with great precision and extremely fast, while being robust during major 
photometric alterations. 

[0083] An embodiment variant of the technical means making It possible to 
perform this detection shall be described in greater detail below with reference to 
Figure 12. 

Second Embodiment Variant 

[0084] In the case of the second embodiment variant of the present invention 
which will now be described with reference to Figures 5, 6, 7 and 8, the system is 
designed for identifying a specific audiovisual sequence 2 within any flow 3 of 
audiovisual sequences 7. 

[0085] The objective is to identify a proprietary audiovisual sequence 5 in the 
flow 3. 

[0086] In the case of this second embodiment variant, the system comprises 
first computer analysis means 40 for extracting reference indexes 10 from the specific 
audiovisual sequence 2, so as to make up a reference set 30 of reference indexes 10. 

[0087] Each reference index 10 of the reference set 30 Is calculated by 

calculation means 38 using an indexing process 39 comparable to the one that was 
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described in the case of the first embodiment variant. Each reference index 10 appears 
in the form of an ordered and finite set 21b of values 20b, and in particular in the form 
of a characteristic vector 9b encoding the content of each specific image 11 of the 
specific audiovisual sequence 2. A reference set 30 characteristic of the specific 
audiovisual sequence 2 is thus obtained. 

[0088] The system additionally comprises reception means 41 for receiving the 
flow 3 of audiovisual sequences 7 capable of comprising at least one specific 
audiovisual sequence 2. 

[0089] The system additionally comprises computer processing means 42 for 
digitizing the flow 3 of audiovisual sequences 7. In the case of this second embodiment 
variant, the technical means used to calculate the current index 14 of each current 
image 13 of an audiovisual sequence 7 will not be described. In fact, they are 
comparable to those described in the case of the first embodiment variant. 

[0090] The system additionally comprises comparison means 44 for comparing 

the reference indexes 10 of the specifics images 11 making up a specific audiovisual 
sequence 2 with the current indexes 14 of the current images 13 of the monitored flow 
3. It is thus possible to detect a specific audiovisual sequence 2 within a flow 3 with 
great precision and extremely fast, while being robust during major photometric 
alterations. 

[0091] An embodiment variant of the technical means malcing it possible to 
perform this detection shall be described in greater detail below with reference to 
Figures 14 and 15. 

[0092] The technical means, in particular the first calculation means 38 and the 
second calculation means 43, used to calculate the reference indexes 10 or to calculate 
the current indexes 14, might be combined in the same computer system; however, 
calculating the reference indexes 10 in systems other than those used to calculate the 
current indexes 14 is not departing from the field of the present invention. This remark 
concerns both the first embodiment variant and the second embodiment variant. 
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Indexing Process 

[0093] In the case of the first embodiment variant of the present invention as in 
the case of the second embodiment variant, the first calculation means 38 and the 
second calculation means 43 for calculating the reference indexes 10 and the current 
indexes 14 use an indexing process, which will now be described with reference to an 
embodiment shown in Figures 9 and 10. The term "Index" was used to designate a 
reference index 10 or a current Index 14 when no distinction was made between them. 

[0094] The elements having comparable functions for calculating the reference 
indexes 10 and the current Indexes 14 were referenced with the same reference 
numbers in the figures. 

[0095] The first calculation means 38 for calculating a reference index 10 of a 

specific Image 11 comprise sampling means 45 for resampling the specific image 11 as 
a resampled specific image with fixed dimensions in advance. This resampled specific 
Image is hereinafter called the normalized specific Image 116. In order to calculate the 
reference Indexes 10, the first calculation means 38 additionally comprise means for 
the discrete quantization 46 of the pixel values of the specific Image 11 resampled 116. 
After discrete quantization, the specific image 11 resampled is represented by a matrix 
19 of the pixel values 17. The first means for calculating 38 the reference index 10 of 
a specific Image 11 additionally comprise sequencing means 47 for arranging the pixel 
values 17 according to a predetermined running order of the positions 18 In the matrix 
19, and In particular by concatenating the values of each line of the matrix in the form 
of a characteristic vector 9a. The reference Index 10 is thus obtained. 

[0096] If the specific image 11 Is a color Image comprising levels of colors, the 
system additionally comprises conversion means 48 for converting the levels of colors 
of the specific Image 11 to be resampled to levels of gray beforehand. 

[0097] Preferably, according to the present invention, the first calculation means 
38 additionally comprise reference processing means 49a for calculating the discrete 
entropy of the distribution of the values of the reference Index 10. This entropy Is 
hereinafter called the reference marginal entropy 50a. 

[0098] The comparison time of two indexes is the time needed to calculate the 
comparison distance between the indexes. By adding this reference marginal entropy 
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value 50a to the reference index 10, the calculation of the comparison distance no 
longer requires the calculation of the reference entropy value 50a. The comparison 
time is thus reduced. 

[0099] It is possible to add this reference marginal entropy value 50a to the 
reference index 10. 

[0100] Preferably also according to the present invention, the second calculation 

means 43 for calculating a current index 14 of a current image 13 comprise sampling 
means 45 for resampling the current image 13 as a current image with fixed 
dimensions in advance. This resampled current image is hereinafter called the 
normalized current image 136. The second calculation means 43 also comprise, for 
calculating a current index 14 of a current image 13, means for the discrete 
quantization 46 of the pixel values of the current Image 13. After discrete quantization, 
the current image 13 resampled Is represented by a matrix 19 of the pixel values 17. 
The second calculation means 43 additionally comprise, for calculating a current index 
14 of a current image 13, sequencing means 47 for arranging the pixel values 
according to a predetermined running order of the positions 18 In the matrix, and in 
particular by concatenating the values of each line of the matrix In the form of a 
characteristic vector 9b. The current index 14 is thus obtained. 

[0101] If the current image 13 is a color image comprising levels of colors, the 
system additionally comprises conversion means 48 for converting the levels of colors 
of the current image 13 to be resampled to levels of gray beforehand. 

[0102] Preferably, according to the present invention, the second calculation 

means 43 additionally comprise current processing means 49b for calculating the 
discrete entropy of the distribution of the values of the current index 14. This entropy 
is hereinafter called the current marginal entropy 50b. 

[0103] It is thus possible to optimize the comparison time. It is possible to add 

this current marginal entropy value 50b to the current index 14. 

[0104] The reference 16 was sometimes used to designate a normalized specific 

image whether it was a normalized specific image 116 or a normalized current image 
136. 
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Calculation of an Index Comparison Distance 

[0105] In the case of the first embodiment variant of the present invention as in 
the case of the second embodiment variant, the system comprises comparison means 
44 for (i) comparing the reference index 10 of the specific image 11 with the current 
index 14 of the current image 13 of the monitored flow 3 or for (ii) comparing the 
reference indexes 10 of the specific images 11 malcing up a specific audiovisual 
sequence 2 with the current indexes 14 of the current images 13 of the monitored flow 
3. 

[0106] An advantageous embodiment of the technical means making it possible 
to perform these comparisons shall now be described with reference to Figures 11 and 
12. For this purpose, the concept of comparison distance 29 in terms of the present 
invention should be explained. 

[0107] Just as this was described above, each reference Index 10 and each 

current Index 14 appear in the form of ordered and finite sets 21a and 21b of values 
20a and 20b. Therefore, it is possible to Identify these values 20a and 20b in the 
reference index 10 and the current index 14 by a system of coordinates 22. 

[0108] The system additionally comprises third calculation means 52 for 
defining, for a given coordinate 24 of the system of coordinates 22, a pair of values 25, 
26, of which the first value 25 is the value appearing in the reference index 10 
associated with the given coordinate 24, and of which the second value 26 is the value 
appearing in the current index 14 associated with the given coordinate 24. 

[0109] The third calculation means 52 make it possible to calculate the 
bidimensional histogram 27 of the pairs of values 25, 26 obtained for all the 
coordinates of the system of coordinates of the reference index 10 and of the current 
index 14. 

[0110] The third calculation means 52 also make it possible to calculate the 

discrete entropy of the bidimensional histogram, hereinafter called the entropy of the 
bidimensional histogram 28. 

[0111] The third calculation means 52 also make it possible to calculate a 

comparison distance 29 between a reference index 10 and a current index 14, forming 
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the ratio between the sum of the reference marginal entropy 50a and of the current 
marginal entropy 50b reduced by the entropy of the bidimensional histogram 28 as the 
numerator and the sum of the reference marginal entropy 50a and of the current 
marginal entropy 50b as the denominator. 

Extraction of Reference Indexes 

[0112] Now that the concept of comparison distance 29 between a reference 
index 10 and a current Index 14 has been explained, it Is possible to complete the 
description of the second variant of the present Invention In the case of an 
advantageous embodiment with reference to Figure 13. In the case of this 
advantageous embodiment, designed to make it possible to detect a specific 
audiovisual sequence 2 within a flow 3 of audiovisual sequences 7, one proceeds 
beforehand with a phase of extracting reference Indexes 10 so as to form a reference 
set 30. 

[0113] To extract the reference indexes 10 of the specific audiovisual sequence 

2 from the specific audiovisual sequence 2, made up of specific images 11, the system 
additionally comprises fourth calculation means 53. These fourth calculation means 53 
use a calculation algorithm 54 comprising a step of initializing a reference set 30 
containing the reference indexes 10 of specific images. The reference set 30 is 
initialized with the reference index 100 of the first specific image 110 of the specific 
audiovisual sequence 2. The reference index 100 of the first specific Image 110 of the 
specific audiovisual sequence 2 constitutes the first reference index of the reference set 
30. The calculation algorithm 54 additionally comprises: 

(a) the step of (I) calculating, for each specific Image 11 of the specific 
audiovisual sequence 2, a temporary current Index 31 and (11) of calculating a 
comparison distance 29 between the temporary current index 31 and the last reference 
index 32 added to the reference set 30, 

(b) the step of comparing the comparison distance 29 between the 
temporary current index 31 and the last reference index 32 added to the reference set 
30 to a predetermined threshold SE 33, 
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(c) the step of adding the temporary current index 31 to the reference 
set 30, if the comparison distance 29 exceeds the predetermined threshold SE 33. 

[0114] The temporary current index 31 thus becomes the last reference index 
32 of the reference set 30. The calculation algorithm 54 additionally comprises the 
step of repeating the steps (a) through (c) up to the end of the specific audiovisual 
sequence 2. 

Detection 

[0115] The final phase of the method of detecting the specific image 11, and in 
particular the proprietary image 4, within any flow 3 of images 6 shall now be 
described in the case of the first embodiment variant with reference to Figure 12. For 
this purpose, the third calculation means 52 compare the comparison distance 29 
between each reference index 10 and the current index 14 of the current image 13 of 
the monitored flow 3 to a predetermined threshold SF 65. The specific image 11 is said 
to be detected within any flow 3 of images 6 when the comparison distance 29 between 
the reference index 10 of the specific image 11 and the current index 14 is less than 
the predetermined threshold SF 65. 

[0116] The final phase of the method of detecting the specific audiovisual 
sequence 2, and in particular the proprietar/ audiovisual sequence 5, within any flow 3 
of audiovisual sequences 7 shall now be described in the case of the second 
embodiment variant with reference to Figures 14 and 15, In this case, the system 
comprises initialization means 57 for loading the value -1, minus one, of a variable T 
34, in a first register T 55, and the value 0, zero, of a variable D 35, in a second 
register D 56. 

[0117] In the case of this embodiment variant, the system additionally 
comprises fifth calculation means 58 for calculating, for each reference index 10 of the 
reference set 30, the comparison distance 29 between the reference index 10 
considered of the reference set 30 and the current index 14 of a current image 13 of 
the monitored flow 3. 

[0118] If the comparison distance 29 thus calculated is less than a 

predetermined threshold SD 59, the second register D 56 is increased by one. This 
condition is hereinafter called the condition for detecting reference Indexes 10. 
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[0119] The moment when the first reference index 10 of the reference set 30 of 
the specific audiovisual sequence 2 meets the detection condition is hereinafter called 
the moment of the first detection. 

[0120] The fifth calculation means 58 are equipped for loading in the first 
register T 55 the time elapsed since the moment of the first detection if the value 
stored in the second register D 56 is different from zero. The fifth calculation means 58 
are equipped (i) for repeating the calculation of the comparison distance 29 until the 
value stored in the second register D 56 reaches the predetermined threshold SD 59, 
or (ii) for repeating the use of the initialization means if the value stored in the first 
register T 55 exceeds a predetermined threshold ST 60. 

[0121] In such a way that the specific audiovisual sequence 2 can be said to be 

detected if the value stored in the second register D 56 reaches the predetermined 
threshold SD 59. 

[0122] Figure 15 shows the flow chart of the algorithm that was just described. 



